Do Summaries Help? A Task-Based Evaluation of Multi-Document Summarization
نویسندگان
چکیده
We describe a task-based evaluation to determine whether multi-document summaries measurably improve user performance when using online news browsing systems for directed research. We evaluated the multi-document summaries generated by Newsblaster, a robust news browsing system that clusters online news articles and summarizes multiple articles on each event. Four groups of subjects were asked to perform the same time-restricted fact-gathering tasks, reading news under different conditions: no summaries at all, single sentence summaries drawn from one of the articles, Newsblaster multi-document summaries, and human summaries. Our results show that, in comparison to source documents only, the quality of reports assembled using Newsblaster summaries was significantly better and user satisfaction was higher with both Newsblaster and human summaries.
منابع مشابه
Using Background Information for Multi-document Summarization and Summaries in Response to a Question
In our second participation to the DUC evaluation, we used the SumUM system for the multidocument summarization focused by events task and summaries in response to a question task. Our multi-document summarization algorithm is based on the use of background information gathered by summarizing previous texts, which is then combined with a new document of the cluster. For producing the summaries ...
متن کاملMulti-document multilingual summarization and evaluation tracks in ACL 2013 MultiLing Workshop
The MultiLing 2013 Workshop of ACL 2013 posed a multi-lingual, multidocument summarization task to the summarization community, aiming to quantify and measure the performance of multi-lingual, multi-document summarization systems across languages. The task was to create a 240–250 word summary from 10 news articles, describing a given topic. The texts of each topic were provided in 10 languages ...
متن کاملUsing N-Grams To Understand the Nature of Summaries
Although single-document summarization is a well-studied task, the nature of multidocument summarization is only beginning to be studied in detail. While close attention has been paid to what technologies are necessary when moving from single to multi-document summarization, the properties of humanwritten multi-document summaries have not been quantified. In this paper, we empirically character...
متن کاملA Graph-based Approach to Cross-language Multi-document Summarization
Cross-language summarization is the task of generating a summary in a language different from the language of the source documents. In this paper, we propose a graph-based approach to multi-document summarization that integrates machine translation quality scores in the sentence extraction process. We evaluate our method on a manually translated subset of the DUC 2004 evaluation campaign. Resul...
متن کاملMulti-Document Summarization: Methodologies and Evaluations
This paper describes a system for the summarization of multiple documents. The system produces multi-document summaries using clustering techniques to identify common themes across the set of documents. For each theme, the system identifies representative passages that are included in the final summary. We also describe a methodology for evaluation of our system which is based upon a question a...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2005